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ABSTRACT 



The invention comprises a method and system for achieving 
increased depth of focus at photography. The method creates 
an improvement of depth of field at use of traditional "Tilt 
and Sniff - technique, and increased possibilities to obtain 
good depth of field with other and quite ordinary cameras. 
The method is primarily meant for electronic cameras, 
where the image is created electronically from a sensor in 
the camera and is based on a multi photo technique. Several 
photos are shot with different focused parts of the scene 
subjects in respective photo, and a basic image is integrated 
by contributions from the different images. Calculated 
image transfers based on lens- or sensor settings for respec- 
tive image, compose a starting point for corrections of 
images and selection of respective image contributions. 

18 Claims, No Drawings 
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INCREASED DEPTH OF FIELD FOR decrease the angle of refraction from the lens. Thus the 

PHOTOGRAPHY image plane can be farther from the real focus, without too 

large blurring errors. However the subject or the light 

BACKGROUND OF THE INVENTION conditions might reduce the possibilities. 

1. Field of the Invention 5 Examples on Prior Art 

The invention comprises a method to increase the depth of [n the field of television the image experienced by the 

focus (or field) for photography, and means for the method. human beings has always been created by the use of many 

The method constitutes an improvement of the depth of "subimages" integrated together. By showing several frames 

focus in use of traditional T / S — technology, "Tilt and 10 (50 to 60) per second, the subimages are integrated in time, 

Shift", and an increased possibility for improved depth of in order to show smooth movements, increase the resolution 

field with other cameras. The method is mainly aimed for and the signal to noise ratio. A study of only one subimage 

digital cameras, where the image is created from a sensor in reveals a poor image quality. An ordinary still camera, in 

the camera, instead of on a photographic film. By use of contrast gives a single shot image of excellent quality, 

present scanning technology however, it is easy to transfer 15 including much higher resolution and very low noise con- 

a film photo to an electronic image. The method can be used tent. 

also for those cases. In the following the method is mainly In the fi eld 0 f rcS earch for high definition television, there 

presented for the cases, when the picture is created elec- nas been an increased awareness of second order effects of 

tronically directly in the camera. mc increase of image resolution. The one that will be 

2. Description of the Related Art 20 discussed here is the aspect of depth of focus. 

In T / S cameras one or both of the lens and the "image For a television image with low resolution, there is a large 

plane" are movable. The image plane is that plane, where the span of subject distances that can be "focused" lo a single 

film or the sensor is positioned. A plane in the scene is image plane, because the blurring of the optics is anyhow 

sharply focused in a "focused plane", which might be less than the resolution of the sensor detectors. However 

localized in front of, back of or cutting the image plane. 25 increasing the sensor pixel resolution e. g. with a factor 4 

Proportional to the distance between the image plane and the means a decrease of the depth of focus with a factor 2. 

focused plane, the blurring grows larger in the image. In T For television it is an obvious method to increase the 

/ S cameras the photographer often views the image on a depth of focus, by using the natural stream of new video 

focusing screen positioned in the image plane. He controls images integrating those together, while changing the focus 

the camera settings for a focused image on the focusing 30 setting, 

screen. Then he places the film or sensor in this position and There are several similar patents in the field of television 

shoots. using this method of increasing the depth of focus. Most of 

Photographers use many different ways of working. The those patents use transfer of the ordinary X,Y -based image 
profession is often creative, sometimes artistic. In practice to a space frequency based image description. They are 
the photographer puts the camera in position directed to the 35 using power comparisons in the space frequency or f re- 
subject, arranges the image plane in the desired perspective, quency filtered band pass domain, between subimages to 
usually vertical, as we are used to view the world from a define which subimage is in focus. They are differing in the 
vertical perspective. Tilting an ordinary camera upwards, e method details of performing this object, 
g for placing a church tower inside the seeker, the camera One method according to Mimura et. al. (U.S. pat. No. 5, 
will produce an image of the tower with a tilting perspective, 40 282,045) uses changing of the focus distance between 
showing a tower tilting or falling in. Parallel vertical lines frames, and then utilizing the integration of subimages in the 
are no more parallel in the image. mind of human viewers. The result is said to increase the 

Then the photographer focuses and positions the subject experienced depth of focus in the mind of humans, 

for the image. Here difficulties might arise, which the 45 The method also includes a "contour corrector", which 

invention can solve. The focal aperture is not only used for function is not described, but is implicitly an ordinary 

control of the light strength, but also for increasing the depth artificial "contour sharpener", said to add higher frequencies 

of field, e g at product photography. Increased exposure can to the contour, which after integration in the mind of 

compensate smaller apertures. However long time of expo- humans, is experienced as a higher resolution image, 

sure implies risks for mechanical disturbances blurring the SQ There are many different problems involved in those 

sharpness. television applications, e g. 

T / S cameras are used for product photography, and when There are time restrictions. Only a short time is available 

there is a desire to increase the depth of field from what an for calculations and mechanical focusing, due to the 

ordinary camera can offer. A close picture on a product with high image frequency. 

large extension in distance, might be given focus both on the 55 By the selection process of focused frames, the number of 

front and back parts, by tilting the focal plane, cutting the integrations decreases and the noise increases, decreas- 

front as well as the back part of the product image. The ing the image quality. 

tilting might be arranged in two ways. One by positioning i n television there are generally movements, by subjects 
the film or sensor in the tilting image plane. The other by m the scene, or the camera panning or zooming the 
tilting the lens instead, turning the focal plane into the 60 scene. In fact the movements are inherent in the tele- 
desired angle. Usually a vertical image plane is desired, vision approach, and necessary for the human expec- 
obtaining a vertical perspective, and then there is just the tations on the television media. Then there are small 
method of tilting the lens, until the focal plane is positioned possibilities for methods utilizing exact matching of 
vertically in the desired image plane. successive subimages with different focus, as the sub- 
As subjects generally are not extended just in a plane of 65 images contains other changes due to movements. In 
two dimensions, but are three-dimensional bodies with the patent of Mimura et al, the depth of focus function 
varying extensions, also focal aperture reductions are used to is simply put off, when there are movements. 
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On the other hand the television is in many ways "for- 
giving" image deficiencies. A poor image is immediately 
followed by new ones in the continuous video-stream of 
images. The short time involved means that many imper- 
fections pass the viewer unnoticed. Then a process of 
selecting focused image parts and combining those together 
can be allowed to involve some inaccuracies and non-perfect 
matches, without disturbing the viewer's total integrated 
experience of the television video-image quality. This is 
opposite to the application of a still camera. The still photo 
is viewed for much longer time periods, and all imperfec- 
tions and small details are also time integrated to noticeable 
and disturbing image quality problems. In the professional 
business, e. g. in case of a new company product photo, there 
is no forgiveness to be expected for even small image quality 
deficiencies. 

BRIEF SUMMARY OF THE INVENTION 

Invention Functions 

The present invention concerns a method to achieve 
sharpness and depth of field for a still photo by use of 
multi-shots, where different camera settings can be used 
including aperture reduction technique. The method utilizes 
possibilities for digital image transfer after the shot, and the 
involved processes might be performed using automatic 
methods to various extents. An operator might also interact 
on computer screens, manually improve and control the 
result of the processes. 

The Method Comprises 

The camera sensor is turned into a suitable focal plane and 
the photo is shot. 

Then the perspective can be turned into a desired position 
by image transformation. One way to do this is sensing the 
sensor turning in relation to the desired perspective and feed 
data to a calculation unit, which calculates how much the 
image should be turned and makes the transformations. E. g. 
if the image is tilted an angle u related to the desired vertical 
plane, then the image plane can be turned around its front 
edge by u. 

Then image points at the distance a from the axis would 
be scaled to the distance a*cos(u) from the axis. Those 
points are also moved closer to the lens, a distance a*sin(u), 
which means the same reduction of scale as moving the 
image the corresponding distance closer to the lens, i. e. a 
distance b to the lens related to (a^sin^+b). This is easily 
illustrated by drawing radiation lines through the optical 
center to respective image point. Thus turning the image 
around the axis, the points on the image will be projected 
through simple geometrical rules to the new image plane. 

The choice of turning axis can be transferred as turning 
around another axis plus a transversal movement. Transver- 
sal movements of image elements, changing the distance 
from the lens, result in a proportional change of scale of the 
image element. Thus ordinary focusing gives rise to size 
changes. Knowledge about the relative image distance from 
the lens (the optical center) would be used for the size scale. 

When there are defocused image parts, also the de focused 
area will be enlarged proportional to a general image 
enlargement. A defocused edge would be wider, if special 
actions are not performed. Such actions are included in the 
invention and will be presented further in the section about 
enlargement. 

Projections of images might be performed as second 
steps, in turning to the right image perspective. The operator 
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might fine tune or control the total process, by introducing 
the desired axis and turning. He can also define a certain 
speed of turn and study the image turning until it reaches the 
desired position, when he can stop the turning or interac- 
5 tively change the turn directions, fine tuning the right 
position. 

The calculations can be performed using a basic image, 
which has been mapped to an increased number of image 
pixels. Then unnecessary losses of resolution can be avoided 

io through the calculations and image processing. 

The control setting of desired focus planes at photography 
can be simplified and automated by the photographer setting 
the focus successively on those points, through which he 
wants the plane of focus. With known coordinates and focus 

15 settings the focus plane can be calculated and any turning of 
lens or sensor also. Simple calculation programs, including 
the camera optics, can be stored in the pocket calculator, 
which can present the result as scale partitions for a turning 
control. With a PC or other computers the operator can work 

20 with the image on the computer screen. Programs in the 
computer can control automatic focus measurements and 
calculation of focal plane and finally control turning and 
movements of the lens (sensor) to the right position. Thus 
the process can be performed as a production process, which 

25 is often wanted. 

Creation of depth of field by use of several "stills". 
At product photography the scene is often static, i. e. 
several photos can be shot reproducible. A product with 

3Q relatively large extension in distance, can be photographed 
from the same position with focus set for different parts for 
the different photos. On the different photos, different parts 
are respective focused and defocused. They are also of 
different size. 

35 Simple addition of image elements. 

A sharp image of the product can be created by adding 
together focused different image elements from respective 
image. The process is made possible by first changing the 
image scale to the same size, according to sections above. In 

40 many situations the operator might do the element selection 
on a computer screen and determine from which image 
respective element shall be transferred to the final image. 

The process can also be automatic according to the 
invented methods. The focus for an image sub-area is 

45 compared between images and the respective focused image 
element is selected. The division borders between neighbor- 
ing image elements are chosen, e g where the defocusing is 
about the same from the different images. Any small defi- 
ciencies in the division interfaces might e g be handled by 

50 averaging that area between the respective images. Simple 
product shapes such as a cereal box, might be photographed 
from a perspective, which gives a corner the shortest dis- 
tance to the camera and other parts successively farther 
distant. A series of photos with successively changed focus 

55 can produce the said focused image elements for image 
element additions. 

The cereal box photographed inclined from a high front 
position, shows three surfaces; a long, a short and an upper 
side. By tilting the sensor or lens, a focused plane might be 

60 positioned along each of the three apparent surfaces. The 
three images are transformed by calculations to the selected 
basic image. If the sensor is turned, then the transformation 
also includes perspective returning. 
After the calculations the product image can be combined 

65 from the three sharp image elements. In practice small 
deviations might have arisen in positions and sizes between 
the three different photos. Disturbances might have shaken 



05/11/2004, EAST Version: 1.4.1 



US 6,4' 

5 

the positions and controls. So sharp borders and edges are 
identified for the different images and common comers and 
boundaries are matched together, through fine tuned correc- 
tions of position and scale for the different images. 

The basic image might e. g. be an image with the right 
perspective e. g. a vertical sensor and a lens setting giving 
a focus plane for a product's front, or some other important 
part of the product. This basic "image" can be defined by 
sensor and lens settings and be the defined "basic image" 
without even being photographed, also if a photo would 
have simplified the matching of the different photos to the 
basic image. 

When additional pictures are desired for the integration of 
a sharp cereal box, then an extra focal plane might be 
positioned along the outmost left and right edges and 
through the diagonal of the upper surface, for helping 
adjustments of the said surfaces. When the table surface is 
of interest, on which the box is standing, then a focal plane 
might be positioned along this surface for an additional 
photo. 

The integration of the final image might in this example 
be relatively simple, as each sharp element is easily identi- 
fied and the defocused areas are substituted by sharp areas 
from other images. 

Recalculations of perspective and scale of size is done to 
the defined basic image format, with the help of known 
settings for the sensor and lens. Analyses afterwards without 
knowledge of the settings, to find out the image position 
relative to another image position, can be performed, but it 
might need some more processing. 

Corrections at the integration of respective sharp image 
elements are performed using common image parts e. g. 
edges and comers. Often there are image and text details, 
which simplify the match. If the adjustments of an image 
wouldn't be good enough, e. g. dependent on a difficult 
perspective change, then local adjustments can be done for 
the different image elements. The sharp details are posi- 
tioned in agreement with information from other images, 
and where distances in between are needed to be increased 
or decreased in a noncoherent way, the local area selected 
therefore is preferable an area with slow light and color 
variations. 

Complex products. 

Products might be more or less complicated in structure. 
There might be arms, poles, holes etc. Those can be iden- 
tified by operators as special areas, "mountains and valleys", 
where a small area might contain large distance variations, 
and whereby several photos might be needed to cover the 
dynamics. Areas with mountains and valleys can also be 
identified automatically according to the invention. They are 
areas, which are characterized in causing strong defocusing 
in consecutive images, although the neighboring image has 
sharp elements and though the transversal distance is close 
between the sharp parts of respective image. The operator 
can be alarmed about such and other problem areas by 
blinking on the screen, possibly at the position and with side 
comments. Often the operator might find focal planes, which 
can be positioned along steep mountains. And with addi- 
tional photos, better integrated sharpness can be obtained for 
such areas. 

If the product has an arm or a pole, then one or more 
images might be shot with focal planes through the arm, 
such that this subject is shown as a sharp object. In a 
corresponding way as for the cereal box, those images are 
transformed to a basic image for integration with the other 
images on the product. 
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Detailed Description of the Invention 

The process for image element identification might start at 
any photo, but it might be an advantage to start with the front 
or back. In this presentation the start is the front photo, and 

S its center element. 

An image area is selected. The area is surveyed consid- 
ering borders, i. e, borderlines and edges with fast variations 
of light strength or color. The color resolution might be 
worse, however. The selection method might be the 

10 derivative, relative derivative (l/p*dp/dq; q=x,y;), differ- 
ence in light or color strength, the relative difference of 
strength, etc. Those borders are compared with correspond- 
ing values for the photos below, and they are defined as 
belonging to the front photo, when the borders have not 

15 better focus in photo number 2. The space in between the 
borders includes surfaces, and surfaces limited by borders 
belonging to photo 1 are also preliminarily assigned to this 
photo until farther steps. A new adjoining image area is 
selected and the process is repeated, until the whole image 

2 0 has been worked through for the given series of photos, 
image 1 to n. 

There are alternative search processes preferable in cer- 
tain cases, e. g. when there are many image elements from 
different images in the selected search area. Then it might be 

25 better directly analyzing edges of the present photos, and by 
comparisons identify the respective belongings of the dif- 
ferent edges. When the following areas been worked 
through, the final image can be integrated successively. 
Borders are selected and processed according to examples. 

30 When a border is an edge between two areas on different 
focused distances, the edge might be assigned to the first 
area, and the transition position between images might be 
selected at the edge on the second image. 

Aspects on Focus and Sharpness 
Sharpness when changing the scale of size, enlargements. 
At enlargements all the image elements are enlarged. 
Those elements with significant distance extensions are 
enlarged including their defocused parts, which then might 

40 be more apparent. Defocused borders, e. g. the border 
between a red subject and a green background, are imaged 
giving a border width containing various red-green mix- 
tures. For the same defocusing degree, the real edge position 
obtains 50% contributions from each side. A defocused point 

45 obtains a blurring defocused circle with smeared contribu- 
tions from the point. Hps, corners and irregular shapes are 
imaged blurred, round smoothed and loosing their detail 
positions. Geometrical objects can be reshaped, by recon- 
struction of the edges to the comer tips etc. 

50 When the lack of sharpness is depending on the focusing 
distances, as in the described examples, the focused state is 
determined by simple optical rules, and according to the 
invention the focused image can be estimated and often also 
the defocused image borders can be corrected. 

55 Digital photography is based on sensors, usually with 
smaller dimensions than the final photo image. The image 
processing includes enlargements and then an interest is not 
loosing sharpness. Especially at product photography the 
subject consists of human designed structures, i. e. identi- 

60 fiable geometrical forms. Then there are preconditions to do 
corrections of defocused well-known detail structures in 
many cases. 

There is also a natural interest in good-looking product 
pictures, which emphasizes the importance of sharpness. 
65 Close up pictures on products enlarge the depth of focus 
problems, and then also the need of methods to improve the 
image quality. 
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Depth of field is relatively short at close ups. By reduction A subject on the distance a+da is focused on the distance 
of the lens focal distance, the subject image decreases b-db. 
approximately proportional, but the relative depth of field Simple optical rules give: 
might be increased. Subsequent magnifications increase the 

blurring too. Photography at distances, using zoom, might 5 Ua+Hb=i/f; (i) 

give the product a flat impression, which might be nega- 
tively experienced. With a small sensor and the product 
object filling the image, there is a need for a lens with a small j n me same wav: 
focal distance, giving an angle of view, when looking at the 

photo, that gives associations to the right appearance of the 10 1/ b{\-db / / (a+da); (3) 

product. Considering that aspect, small products should be 

shot at short distances with short focal distances. (2) in (3) gives: 

The invention includes methods, based on knowledge db / b=da*f 1 a 2 * v (1- fla+dafa) (4) 

about optical rules, for correction of defocusing, and it is 
based on knowledge about geometry for estimating struc- 35 and 
hires and improving the sharpness. 

Correction of sharpness using optic methods. 

Above, some methods have been presented, by which the proportionality gives: 
image sharpness can be improved e. g. at product photog- 2Q 

raphy. Below there is a summary: r t =r 0 * db/bj (6) 

a. Integration of several images on a product, being shot with 

different camera settings, including focus. Perspective (5) m (6) gives: 
turning and scale sizing can be used to match the different 
images to get a sharper image, ^ 

b. Adding information by use of photo with extra small focal For the same sub j ect distance (a) and depth of subject (da) 
aperture. By that the light will be extra reduced and the it ^ obtained that the defocusing is decreasing, when the 
depth of field increased. Then object details with high aperture (r 0 ) and the focal distance (f) are decreasing. When 
light and contrast are imaged with larger depth of field. f decreases, then the image also decreases, however. The 
Such images can be used to match different images with 3Q re lative unsharpness is: 

less depth of field e g in the method in point (a) above. 

c. Adding information from photos being shot with less focal rj h-r Q * da/ (8) 



db!b~da*fl a 2 ; (5) 



distances. Changing the scale is simplified when the 
position is maintained of the subject relative the center of 



rj h -/* daf (H*a*2Q); (9) 



the lens. Depth of field might be improved also in those 35 wfaere ^ aperture r . 

cases. On the other hand, the image object is smaller and ^ tbe rcducUon of the focal aperture fa m important 

is not utilizing the resolution at maximum. Also this type ibility for increase of me depth of field , apert ure 

of photos can be used in the matching of images. cannot however be reduced t00 much> L e> the light level 

d. By tilting the lens or sensor, a corresponding tilting focal arriving a( me gensor must be lafge enQUgh ^ density of 
plane is positioned through the product. The image per- 4Q light for the ^ at the XJSSQX is proport i 0 nal to the 
spective is changed if the sensor is tilted, whereby a dens[ of ^ ftom ^ subjectj I(H) according to: 
subsequent perspective turauig is performed. This type of 

images can be used to give information about sharpness of i(hyi(ff)*H 2 *r 0 2 1 {h*af ; (10) 
elements, which might be missing in point (a). 

e. Consecutive photos can be used for optical calculations of ^ With (10) in (9) it is obtained: 
the focused image distance of an image element and the 

real dimensions in this focused position. This method can ^ J W* * da * fl 

be used e. g. for image elements that are out of focus on L e tf I(h) cannot be funhef reduoed and ^ Ught conditions 

both neighbormg images. and level are selected, then a reduction of the focal distance 

f. By measuring the defocusing of edges and borders the 5Q ^ a ibflit ^ to ^ lhe relative focusingi 
focused position of those image elements can be calcu- Corrections at integration of the new basic image from the 
lated and the border blurring be corrected. { hotQS 

g. Geometric figures can be inter- and extrapolated from ?hotQ ^ ^ ^ ih& same felative position of ^ lens 
consecutive photos and thereby sharpness can be esti- q{ ^ ^ afe simple tQ 
mated for defocused distances. The method might be 5S taasSam b perspective turnings and sizing, knowing the 
simplified by storing a library of a number of usual gettm for ^ kns ^ sensor of respective photo> By a 
geometric figures, and used in comparisons with the lraditiona i camera> foca i Stance settings are the only focal 
image information. Various methods, e.g. best correlation p]m scUing avai]able ^ means that thc photo serics 

might be used. obtains parallel image planes, and no cutting focal planes for 

Some optical relations regarding depth of field. 6Q ^ &&mc sccne yicw Togcther ^ thc othcr applicable 

Usual optical notations and definitions: methods, described above, also those cameras can be used 

a, b, f are distances to the lens from the subject, image, fof advanccd pro duct photography, where depth of field and 

focus point. perspective otherwise gives problems. 

H, h are the dimensions (e. g. the height) of the subject, At multi-photo integration, small deficiencies might arise 

its image. 65 in practice. Tolerances in settings and mechanical distur- 

r 0 , r,- are the radius of the focal aperture, the radius of the bances and imperfections might cause changes between 

defocused circle related to a sharp point. subsequent image shots. Photography with long exposure 
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time might require extra stable basis for camera and 
subjects, preventing disturbances from blurring the image. 
Until further actions, it is presently assumed that each useful 
image has good focus within its limited depth of field. Still 
corrections might be needed, after that the images princi- S 
pally have been transformed to the basic image format. 
Although each image mainly is in the right position, right 
perspective and right size. The following methods can be 
used for subsequent corrections: 

a. Consecutive images with different focus can be corrected, 11 
when a sharp element in one image corresponds to an 
unfocused one in neighboring images. The position of the 
unfocused element can be determined, and by compari- 
sons of detail positions in other images, positions, size of 
scale etc. can be coherently corrected. Common informa- i 
tion for images is used. Calculation processes are faster, 
when a few well-defined elements are selected for the 
comparison process and the elements preferably spread 
out over the image area. For the right image size the 
matching might be reduced to match the positions of the 21 
image elements. For errors in the scale size, the errors of 
scale might be obtained by "measurements" (calculations) 
of the distances between element positions. At perspective 
turning the scale of size might be different in different 
parts of the image, e. g. different in the x- and y directions. 2 

b. Common sharp image elements are obtained from cutting 
focal planes, e. g. for the example of the cereal box in 
point 2.1. By overlapping of common borders and edges 
the different images can be corrected and integrated. 

c. Added images can be shot, positioned at cutting focal 3 
planes e. g. to obtain sharp image elements in a tilted 
plane, which then can be recognized as sharp elements in 
different images. Then common references are obtained 
for and through several images. 

d. Images being shot with significantly reduced aperture 3 
obtain larger depth of field and can be references in 
corrections of other images, especially for elements with 
high light and contrast strength. 

e. Photos being shot with reduced focal distance can also 
give larger depth of field in spite of subsequent magnifi- a 
cations. 

Other examples of the method. 

The sharpness of a product image can also be improved by 
increasing the resolution of the image in added photos, e. g. 
a product might be photographed with only parts of it filling 4 
the image (sensor) area. Then the sensor resolution is better 
utilized for this part of the total product image. The other 
parts of the product can be photographed with the same or 
different magnifications. Then the different images can be 
transformed to a basic image, where the whole desired 5 
image is included, and then with a high resolution. The 
resolution in the subsequent processing of the image is not 
limited by the condition that the whole image must be 
positioned on the sensor. The computer capacity can be used, 
creating a computer image with much higher resolution than 5 
the sensor. The method to integrate the different photos is 
easier, when as earlier the position of the lens optical center 
is mainly maintained relative the image subject. E g that can 
be performed by photographing, turning the camera around 
the lens optical center. In the same way as earlier the < 
integration is simplified when the different images are 
overlapped, where well defined elements are contained, and 
the overlap can also be used in comparisons between images 
for e g corrections. One or more images can also be shot with 
a lens with less focal distance, where essential parts of the < 
whole image are contained, and contribute with general 
information about the basic image. 
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The method of selecting parts of the subject and photo- 
graphing those with larger focal distances, possibly with the 
help of zoom, can be used together with the other methods, 
which are used for increased depth of field. The method 
might be applied for imaging very small details in a product, 
or for details used in the integration of image areas giving 
better resolution and by that a more accurate match. 

Examples on means for performing the improved depth of 
field. 

) In one preferred embodiment, the means consists of units 
in the camera and units in computation equipment, e. g. a 
computer of PC type. The camera units consist of controls, 
which settings can be registered. The settings are for the lens 
and/or the sensor. For traditional cameras with electronic 

; sensor, the distance focusing is made by the lens. T/S 
cameras might have controls for both the lens and the sensor, 
and those have generally large variability, i. e, they can also 
do tilting and give focusing and perspectives in many 
various planes. The settings can be controlled and read, and 

3 the lens and sensor might also, according to the invention, be 
related to the geometry of the camera design. The values on 
the optical system as focal distance, optical center or cor- 
responding lens descriptions might be valuable in subse- 
quent calculations after the photographing. It is simplifying 

5 when the position of the optical center is known, especially 
at lens changes between photos. 

In the calculation unit, the computer, there is software, 
which can store several images and work with transforma- 
tions and transfer of images by size of scale, perspective 

0 turning and positioning. If the optics and camera setting data 
are not included together with the image data transfer, the 
operator or photographer might read and feed in the values 
of the settings, or the computer makes the evaluation. 
The choice of photo set up is made by the photographer, 

5 and he can set the camera manually or with help of the 
computer system. He can e. g. read the position for the point 
he will focus and read the connected focus setting. By 
reading three points a focus plane is defined, and a computer 
can help calculating the settings of lens or sensor to establish 

d the desired focal plane. The computer can also contain an 
additional program for automating all or various parts of the 
work. The control can also include selection of aperture 
from light measurements and other control of camera func- 
tions. 

5 There are also included programs, which select image 
areas from respective transferred image, possibly with inter- 
actions from an operator. 

Programs can be added that, possibly in interaction with 
an operator, select elements from different images, which are 
0 the basis for calculations of subsequent corrections of the 
images, including matching the basic image format for 
integration of focused image elements to the new focused 
basic image. 

What is claimed is: 
5 1. A method to increase the depth of focus, giving 
improved sharpness of subjects with such extensions that 
sharpness regularly cannot be obtained over the whole 
picture, using at least two images obtained with different 
camera settings, which include at least one of change of 
0 focus and change of aperture, in mainly keeping the same 
position of the lens optical center relative to the subject, and 
combining information from those images to obtain one new 
image, the method comprising the following steps: 
a. taking photos of subjects with different focus settings 
5 between images, creating different focus for different 
image areas, including natural variations and distur- 
bances of the photo sequence, generating changes in 
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the images exceeding the said change of focus, the 
changes in images composing changes including any of 
position, scale and perspective; 
b. matching the different photo-images to the position, 
scale and perspective of a selected basic image com- s 
prising the sub-steps of 
bl. selecting a basic image; 

b2. identifying and localizing border lines and highly 
contrasting structures in a number of images; 

b3. calculating the geometrical transfer of a number of 30 
images to the basic image, using at least one of 
position shifts, change of scale and perspective tilt- 
ing; 

b4. choosing common references for evaluating and/or 
correcting the matching of the images using the 
positions of identified common structures; c. build- 15 
ing a new image from the matched images by 
sub-steps including 
cl. analyzing image areas with continuous change of 
focus of image elements in the area, corresponding 
to a continuous change of distance, where focus 20 
changes are utilized for borderlines and/or highly 
contrasting structures in distance, and 
identifying transfer positions, where change of focus 
is of corresponding size seen from the respective 
image focused state, and 25 
selection of a transfer border between image infor- 
mation from one image and next neighboring 
focused image, considering the said transfer 
positions, and 
when large discontinuities in the image transfer are 30 
present, a fine tuned match is performed of the two 
images along the common transfer border, includ- 
ing any method step from (b); 
cZ selecting border lines for analyzing a number of 
border line areas, regarding differences in focus 35 
between the two sides of the border line for the 
respective focused image, for identifying transfer 
borders with focus distance gaps larger than a given 
value; 

correcting the border line area, where the border line 40 
is a transfer border according to (c2), between 
different focused images, with reduction of defo- 
cused image information from the de focused 
image side to the focused image side, and when 
the said focus gap is less than the said given value, 45 
the transfer is considered as the transfer according 
to(cl); 

c3. utilizing, for the analysis of focus in (cl) and (c2) 
fast variations of tight and/or color as functions of 
area based space coordinates, comparisons for best 50 
focused state between images, using borderlines 
and/or highly contrasting structures of subjects, 
d, merging the identified focused image regions, accord- 
ing to (a)-(c), into the said one new image. 

2. A method according to claim 1, further comprising: 55 
analyzing the speed of variation in light or color, selectively 
using derivatives of, relative derivatives of, differences of, 
and combinations or quotients thereof and of the strength of 
light or colors. 

3, A method according to claim 1, comprising: adding 60 
information by use of photo including at least one of (a) and 
(b) below, 

a. adding photo with extra small focal aperture, increasing 
the depth of field and used for corrections of other 
images or matching of other images, 65 

b. adding photo with a cutting focal plane, tilted related to 
another focal plane, obtaining sharp image elements in 
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the tilted plane, which can be recognized in other 
images and used for corrections of other images or 
matching of other images. 

4. A method according to claim 1, comprising: identifying 
borders, which constitute contours between objects belong- 
ing to different focused image distances, where a first object 
is an image of a first subject, which is closer to the lens than 
a second subject, which is reproduced as a second object, 
and where the first object is focused in a first image at a first 
focused image distance with a border to the defocused 
second object, and where the first object is presented in a 
second image at a second focused image distance of the 
second object, where the first object is defocused and its 
blurred image is spread over the said border and disturbing 
the focused image of the focused second object at an area 
along the border, and using at least one of the steps (a) to (d) 
below; 

a. reducing the said disturbance in the second image by 
calculating the disturbance from the image information 
of the focused first object in the first image, and then 
subtracting that disturbance from the second image, 

b. in the second image, the second object has got its 
focused state determined by optical rules, and in the 
said disturbed area along the border the second object 
has a successively reduced intensity towards the said 
border, which intensity is corrected, and the said border 
in the first image is the border, when integrating the 
second object from the second image in the process to 
create the final image, 

c. building a final basic image starting with the said first 
object in the said first image, the image area is 
increased while comparing images for neighboring 
distances, i e comparing the first image with the second 
image, and when passing the disturbed area along the 
border in the second image, the second image contains 
a less disturbed focused image of the second object and 
the said final image area is growing by integration, 
using element areas from the less disturbed second 
object in the second image, 

d. structures of the focused, undisturbed, part of the 
second object in the second image are extrapolated into 
the said disturbed area along the border to achieve 
focusing in this same area along the border, where the 
second object is defocused in the first image. 

5. A method according to claim 1, farther comprising: 
building the new basic image successively in the steps of 

starting at a selected focused image element of a 

selected image; 
increasing the examined area continuously, while com- 
paring with neighboring images; 
making transition of images, when element grows 

sharper at another image; 
increasing the new basic image area with the area from 

the present image; 
selectively starting on the front image and successively 
working towards the background. 

6. A method according to claim 1, further comprising: 
including to the computer means a man machine 

interface, MMI, for interactions with an operator; 
reducing the complexity of the image processing by an 
operator selectively performing steps of 
defining on the computer screen, which parts from 
which image to be transferred to the new basic 
image, reducing the processing to mainly image 
matching and final corrections of the parts in position 
and border defocusing; 
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selecting details from different images for the image 
matching; 

selectively defining strategies for different areas for the 
new image building. 

7. A method according to claim 1, comprising: determin- 
ing the defocused width of borders, edges etc. and subse- 
quently correcting the sharpness also for other image 
elements, selectively by also estimating the image positions 
of the sharp focused elements. 

8. A method according to claim 1, comprising: using 
consecutive images, calculating the image plane distance of 
an image element and its real focused extension at that 
plane, using that for image elements that are defocused in 
images at both side of the theoretically sharp image plane. 

9. A method according to claim 1, comprising: inter- or 
extrapolating geometrical structures, lines, rectangles, 
circles etc. from neighboring images to achieve focusing in 
defocused spaces in between, and selectively expanding the 
method to include storing of geometrical structures in a 
library for comparisons with obtained image elements, 

e. g. by use of correlation, and when good agreements, the 
geometrical structures are used for corrections, the 
geometric structures possibly also including signs, let- 
ters or symbols. 

10. A method according to claim 1, comprising: correcting 
at least one of position, scale, perspective for different 
images, performing position comparisons between corre- 
sponding image elements in at least two images, and selec- 
tively using successive comparisons between different 
images, achieving several images related to each other. 

11. A method according to claim 1, utilizing the sensor 
resolution better, by photographing parts of a product with 
larger focal distance, the product part covering a larger part 
of the sensor area, comprising: photographing other parts of 
the product with the same or different magnification, where 
also image distances related to the focal distance might vary, 
and that different images are transferred to a basic image, 
where the whole image is covered and then with a corre- 
sponding higher resolution, where the capacity of the com- 
puter is utilized, which can give much higher resolution than 
that of the sensor, covering the whole final image, and 
selectively integrating different images, shot while keeping 
the position of the lens optical center relative to the subject, 
e. g. performing that, by turning the camera with the optical 
lens center as the turning point, and simplifying the image 
matching by overlapping images, where well defined ele- 
ments are included and used at comparisons between images 
for possible corrections, and photographing one or more 
images with reduced focal distances, where essential parts of 
the total image are included in the same photo. 

12. Means for performing the tasks according to claim 1, 
comprising: control settings for the camera, calculation unit 
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for storing of image data and software program for process- 
ing of several images and for calculation of transfer of 
images through at least one of; scale of size, perspective and 
position, and for selecting image elements to be used from 
5 respective image and integrating those element to a basic 
image. 

13. Means according to claim 12, aiding the photographer 
or operator with controls of the: means for defining positions 
of points in the image, means for focusing those points and 
register the focus in the calculation unit, means in the 
calculation unit for calculating subsequent corresponding 
values for lens and/or sensor settings, means for control of 
the settings for photographing, and selectively adding means 
for storing basic data for respective control in the calculation 
unit, being e. g. a pocket calculator or a PC, for use at 

15 calculations and calibrations of values for control of the 
control means. 

14. Means according to claim 13, comprising: means for 
the photographer to mark points on a computer screen 
image, and computer programs calculating and controlling 

20 the camera settings automatically by electronic controls. 

15. Means according to claim 12, comprising: control 
settings for at least one of the lens and sensor, relating to 
respective position in the camera, and calculation unit with 
algorithms, controlling settings of respective image photo 

25 and transforming images to a selected basic image, and 
possibly a program unit interacting with an operator for at 
least one of; camera settings for the said image photos, 
image element selection from respective said image and 
integration of image elements to a basic image. 

30 16. A method according to claim 1, where photography 
film are used instead of electronic sensors, comprising; 
photographing in the corresponding way, and subsequently 
scanning the film photos to obtain electronically stored 
images, which are subsequently processed in the corre- 

35 sponding way. 

17. A method according to claim 1, comprising: adding 
photo with reduced focal distance, selectively by zoom or 
change of lens, where image transformations are simplified 
by keeping the position of the lens optical centers relative 

40 the subject, giving a smaller image, not fully utilizing the 
sensor resolution, while depth of field has a potential to be 
increased, although by remagnification of the image also the 
defocused areas are magnified, the defocused elements 
obtaining improved focus and the image being used for 

45 adding information and to help matching other images. 

18. A method according to claim 1, comprising: first 
taking photos of the same image, but with focus on different 
elements of the image with the help of zoomtechnique and 
then using a method of said claims for giving the said same 

50 image an increased depth of field. 

* * * * * 
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